<!DOCTYPE html>
<html lang="en-us">
<head><head>
    <meta name="referrer" content="no-referrer"/>
    <meta name="google-site-verification" content="9vIieCe-Qpd78QOmBl63rGtIVbhY6sYyuxX3j8XWBA4" />
    <meta name="baidu-site-verification" content="LRrmH41lz7" />
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="google-site-verification" content="xBT4GhYoi5qRD5tr338pgPM5OWHHIDR6mNg1a3euekI" />
    <meta name="viewport" content="width=device-width, initial-scale=1">
    
    <meta name="description" content="针对谷歌开源序列化框架protobuf的协议介绍及性能实测">
    
    <meta name="keyword"  content="暴走的初号机, shinji3887, 暴走的初号机的网络日志, 暴走的初号机的博客, shinji3887 Blog, 博客, 个人网站, 互联网, Web, 云原生, PaaS, Istio, Kubernetes, 微服务, Microservice">
    <link rel="shortcut icon" href="/img/favicon.ico">

    <title>protobuf协议介绍及性能实测-同步率400%</title>

    <link rel="canonical" href="/post/protobuf-introduce-and-test/">

    <link rel="stylesheet" href="https://lupeier.cn-sh2.ufileos.com/iDisqus.min.css"/>
	
    
    <link rel="stylesheet" href="https://lupeier.cn-sh2.ufileos.com/bootstrap.min.css">

    
    <link rel="stylesheet" href="https://lupeier.cn-sh2.ufileos.com/hux-blog.min.css">

    
    <link rel="stylesheet" href="https://lupeier.cn-sh2.ufileos.com/syntax.css">

    
    <link rel="stylesheet" href="https://lupeier.cn-sh2.ufileos.com/zanshang.css">

    
    <link href="/css/font-awesome.min.css" rel="stylesheet" type="text/css">
    
    
    <script src="https://lupeier.cn-sh2.ufileos.com/jquery.min.js"></script>
    
    
    <script src="https://lupeier.cn-sh2.ufileos.com/bootstrap.min.js"></script>
    
    
    <script src="https://lupeier.cn-sh2.ufileos.com/hux-blog.min.js"></script>
</head>
</head>

<nav class="navbar navbar-default navbar-custom navbar-fixed-top">
    <div class="container-fluid">
        
        <div class="navbar-header page-scroll">
            <button type="button" class="navbar-toggle">
                <span class="sr-only">Toggle navigation</span>
                <span class="icon-bar"></span>
                <span class="icon-bar"></span>
                <span class="icon-bar"></span>
            </button>
            <a class="navbar-brand" href="/">L&#39;s Blog</a>
        </div>

        
        
        <div id="huxblog_navbar">
            <div class="navbar-collapse">
                <ul class="nav navbar-nav navbar-right">
                    <li>
                        <a href="/">Home</a>
                    </li>
                    
                    <li>
                        <a href="categories/tech">tech</a>
                    </li>
                    
                    <li>
                        <a href="categories/tips">tips</a>
                    </li>
                    
                    <li>
                        <a href="/about">About</a>
                    </li>
                    
                </ul>
            </div>
        </div>
        
    </div>
    
</nav>
<script>
    
    
    
    var $body   = document.body;
    var $toggle = document.querySelector('.navbar-toggle');
    var $navbar = document.querySelector('#huxblog_navbar');
    var $collapse = document.querySelector('.navbar-collapse');

    $toggle.addEventListener('click', handleMagic)
    function handleMagic(e){
        if ($navbar.className.indexOf('in') > 0) {
        
            $navbar.className = " ";
            
            setTimeout(function(){
                
                if($navbar.className.indexOf('in') < 0) {
                    $collapse.style.height = "0px"
                }
            },400)
        }else{
        
            $collapse.style.height = "auto"
            $navbar.className += " in";
        }
    }
</script>




<style type="text/css">
    header.intro-header{
        background-image: url('https://lupeier.cn-sh2.ufileos.com/bandwidth-close-up-connection-1148820.jpg')
    }
</style>
<header class="intro-header" >
    <div class="container">
        <div class="row">
            <div class="col-lg-8 col-lg-offset-2 col-md-10 col-md-offset-1">
                <div class="post-heading">
                    <div class="tags">
                       
                       <a class="tag" href="/tags/microservice" title="Microservice">
                           Microservice
                        </a>
                        
                       <a class="tag" href="/tags/protobuf" title="protobuf">
                           protobuf
                        </a>
                        
                    </div>
                    <h1>protobuf协议介绍及性能实测</h1>
                    <h2 class="subheading"></h2>
                    <span  class="meta">Posted by L&#39; on Monday, December 3, 2018
                        
                    </span>
					<br>
                </div>
            </div>
        </div>
    </div>
</header>




<article>
    <div class="container">
        <div class="row">

            
            <div class="
                col-lg-8 col-lg-offset-2
                col-md-10 col-md-offset-1
                post-container">

        		
                        <header>
                        <h2>TOC</h2>
                        </header>
                        <nav id="TableOfContents">
  <ul>
    <li>
      <ul>
        <li><a href="#协议简介">协议简介</a></li>
        <li><a href="#性能测试">性能测试</a></li>
        <li><a href="#结论">结论</a></li>
        <li><a href="#补充说明">补充说明</a></li>
      </ul>
    </li>
  </ul>
</nav>
        		
        		<blockquote>
<p>protobuf是谷歌开源的一款高性能序列化框架，特点是性能优异，数据结构设计优秀并具有良好的可扩展性，并且配合官方的java、python、go、c++的sdk，可以轻松做到跨语言。本文给出protobuf协议的简单介绍以及与其他框架对比的性能测试结果。</p>
</blockquote>
<h3 id="协议简介">协议简介</h3>
<p>Protocol Buffers 是一种轻便高效的结构化数据存储格式，可以用于结构化数据串行化，或者说序列化。它很适合做数据存储或数据交换格式。可用于通讯协议、数据存储等领域的语言无关、平台无关、可扩展的序列化结构数据格式。你可以理解为另外一种形式的xml，当然protobuf为了追求性能，可读性没有xml或者json那么好，换来的是编码后的报文容量大大缩小以及序列化速度的提高。
要使用使用protobuf，首先需要定义一个.proto格式的文件，格式类似下面这样</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-proto" data-lang="proto">syntax <span style="color:#f92672">=</span> <span style="color:#e6db74">&#34;proto3&#34;</span>;<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span><span style="color:#66d9ef">message</span> <span style="color:#a6e22e">Person</span> {<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>  <span style="color:#66d9ef">required</span> <span style="color:#66d9ef">string</span> name <span style="color:#f92672">=</span> <span style="color:#ae81ff">1</span>;<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>  <span style="color:#66d9ef">required</span> <span style="color:#66d9ef">int32</span> id <span style="color:#f92672">=</span> <span style="color:#ae81ff">2</span>;<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>  <span style="color:#66d9ef">optional</span> <span style="color:#66d9ef">string</span> email <span style="color:#f92672">=</span> <span style="color:#ae81ff">3</span>;<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>  <span style="color:#66d9ef">enum</span> PhoneType {<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>    MOBILE <span style="color:#f92672">=</span> <span style="color:#ae81ff">0</span>;<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>    HOME <span style="color:#f92672">=</span> <span style="color:#ae81ff">1</span>;<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>    WORK <span style="color:#f92672">=</span> <span style="color:#ae81ff">2</span>;<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>  }<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>  <span style="color:#66d9ef">message</span> <span style="color:#a6e22e">PhoneNumber</span> {<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>    <span style="color:#66d9ef">required</span> <span style="color:#66d9ef">string</span> number <span style="color:#f92672">=</span> <span style="color:#ae81ff">1</span>;<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>    <span style="color:#66d9ef">optional</span> PhoneType type <span style="color:#f92672">=</span> <span style="color:#ae81ff">2</span> [<span style="color:#66d9ef">default</span> <span style="color:#f92672">=</span> HOME];<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>  }<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>  <span style="color:#66d9ef">repeated</span> PhoneNumber phone <span style="color:#f92672">=</span> <span style="color:#ae81ff">4</span>;<span style="color:#960050;background-color:#1e0010">
</span><span style="color:#960050;background-color:#1e0010"></span>}<span style="color:#960050;background-color:#1e0010">
</span></code></pre></div><p>目前有v2和v3两种版本，api会略有不同。
修饰符：</p>
<ul>
<li>required : 　不可以增加或删除的字段，必须初始化；</li>
<li>optional : 　 可选字段，可删除，可以不初始化；</li>
<li>repeated : 　可重复字段， 对应到java文件里，生成的是List。</li>
</ul>
<p>更多介绍可参考官网。
执行</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">protoc ./message.proto --java_out<span style="color:#f92672">=</span>./
</code></pre></div><p>可在当前目录下生成对应对应语言的对象描述代码，这边对应的是java的class文件，再把文件拷贝到项目工程目录内，就可以使用了。（需要下载对应语言的protoc二进制程序）。
这边是一个简单的proto报文生成过程
<img src="http://lupeier.cn-sh2.ufileos.com/proto.jpg" alt="timg.jpg"></p>
<p>大家如果之前用过web service的话，应该会有似曾相识的感觉。这个proto文件，我理解为类似于SOAP的wsdl描述文件，即是一份用来描述数据结构的标准文档说明，各语言的sdk可根据该语言的proto文件生成标准的类文件（是不是想起来了wsdl2java），从而达到跨语言的远程调用（RPC调用）。
然而实际使用中，基于这种模式使用还是比较麻烦，如果对象多了要写一堆proto定义文件，另外生成出来的java对象可读性也比较差，和平时用的pojo有很大不同。本文介绍一种更加方便的使用protobuf的方法，就是protostuff。利用这个框架，可以跳过编写proto文件的步骤，直接生成protobuf格式的报文，接收端也可以直接使用该框架将二进制反序列化为Object对象，用到的就是我们平时使用的普通的java对象。利用Protostuff-Runtime模块可以不需要静态编译protoc，只要在runtime的时候传入schema就可以了。下面就来实操一下
首先引入maven依赖</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-xml" data-lang="xml"><span style="color:#f92672">&lt;dependency&gt;</span>
       <span style="color:#f92672">&lt;groupId&gt;</span>io.protostuff<span style="color:#f92672">&lt;/groupId&gt;</span>
       <span style="color:#f92672">&lt;artifactId&gt;</span>protostuff-core<span style="color:#f92672">&lt;/artifactId&gt;</span>
       <span style="color:#f92672">&lt;version&gt;</span>1.6.0<span style="color:#f92672">&lt;/version&gt;</span>
<span style="color:#f92672">&lt;/dependency&gt;</span>
<span style="color:#f92672">&lt;dependency&gt;</span>
       <span style="color:#f92672">&lt;groupId&gt;</span>io.protostuff<span style="color:#f92672">&lt;/groupId&gt;</span>
       <span style="color:#f92672">&lt;artifactId&gt;</span>protostuff-runtime<span style="color:#f92672">&lt;/artifactId&gt;</span>
       <span style="color:#f92672">&lt;version&gt;</span>1.6.0<span style="color:#f92672">&lt;/version&gt;</span>
<span style="color:#f92672">&lt;/dependency&gt;</span>
</code></pre></div><p>编写java对象，这边为了测试写了一个比较复杂的属性带一个循环列表的对象</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java"><span style="color:#f92672">package</span> com.bocsh.proto<span style="color:#f92672">;</span>

<span style="color:#f92672">import</span> java.util.List<span style="color:#f92672">;</span>

<span style="color:#66d9ef">public</span> <span style="color:#66d9ef">class</span> <span style="color:#a6e22e">User</span> <span style="color:#f92672">{</span>

	<span style="color:#66d9ef">private</span> String id<span style="color:#f92672">;</span>

    <span style="color:#66d9ef">private</span> String name<span style="color:#f92672">;</span>

    <span style="color:#66d9ef">private</span> Integer age<span style="color:#f92672">;</span>

    <span style="color:#66d9ef">private</span> String desc<span style="color:#f92672">;</span>
    
    <span style="color:#66d9ef">private</span> List<span style="color:#f92672">&lt;</span>Role<span style="color:#f92672">&gt;</span> roleList<span style="color:#f92672">;</span>

   <span style="color:#75715e">//setter getter  略..
</span><span style="color:#75715e"></span>	<span style="color:#a6e22e">@Override</span>
    <span style="color:#66d9ef">public</span> String <span style="color:#a6e22e">toString</span><span style="color:#f92672">()</span> <span style="color:#f92672">{</span>
        <span style="color:#66d9ef">return</span> <span style="color:#e6db74">&#34;name=&#34;</span> <span style="color:#f92672">+</span> name <span style="color:#f92672">+</span> <span style="color:#e6db74">&#34;,id=&#34;</span> <span style="color:#f92672">+</span> id <span style="color:#f92672">+</span> <span style="color:#e6db74">&#34;,age=&#34;</span> <span style="color:#f92672">+</span> age <span style="color:#f92672">+</span> 
        		<span style="color:#e6db74">&#34;,role1=&#34;</span> <span style="color:#f92672">+</span> roleList<span style="color:#f92672">.</span><span style="color:#a6e22e">get</span><span style="color:#f92672">(</span>0<span style="color:#f92672">).</span><span style="color:#a6e22e">getId</span><span style="color:#f92672">()</span> <span style="color:#f92672">+</span> 
        		<span style="color:#e6db74">&#34;,role2=&#34;</span> <span style="color:#f92672">+</span> roleList<span style="color:#f92672">.</span><span style="color:#a6e22e">get</span><span style="color:#f92672">(</span>1<span style="color:#f92672">).</span><span style="color:#a6e22e">getId</span><span style="color:#f92672">();</span>
    <span style="color:#f92672">}</span>

<span style="color:#f92672">}</span>
</code></pre></div><div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java"><span style="color:#f92672">package</span> com.bocsh.proto<span style="color:#f92672">;</span>

<span style="color:#66d9ef">public</span> <span style="color:#66d9ef">class</span> <span style="color:#a6e22e">Role</span> <span style="color:#f92672">{</span>
	
    <span style="color:#66d9ef">private</span> String id<span style="color:#f92672">;</span>

    <span style="color:#66d9ef">private</span> String name<span style="color:#f92672">;</span>

    <span style="color:#66d9ef">private</span> String desc<span style="color:#f92672">;</span>

    <span style="color:#75715e">//setter getter  略..
</span><span style="color:#75715e"></span>
<span style="color:#f92672">}</span>
</code></pre></div><p>编写测试类进行测试</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-java" data-lang="java"><span style="color:#66d9ef">public</span> <span style="color:#66d9ef">class</span> <span style="color:#a6e22e">ProtoBufUtilTest</span> <span style="color:#f92672">{</span>
	 
    <span style="color:#66d9ef">public</span> <span style="color:#66d9ef">static</span> <span style="color:#66d9ef">void</span> <span style="color:#a6e22e">main</span><span style="color:#f92672">(</span>String<span style="color:#f92672">[]</span> args<span style="color:#f92672">)</span> <span style="color:#66d9ef">throws</span> Exception <span style="color:#f92672">{</span>
 
        User user <span style="color:#f92672">=</span> <span style="color:#66d9ef">new</span> User<span style="color:#f92672">();</span>
        user<span style="color:#f92672">.</span><span style="color:#a6e22e">setAge</span><span style="color:#f92672">(</span>300<span style="color:#f92672">);</span>
        user<span style="color:#f92672">.</span><span style="color:#a6e22e">setDesc</span><span style="color:#f92672">(</span><span style="color:#e6db74">&#34;备注&#34;</span><span style="color:#f92672">);</span>
        user<span style="color:#f92672">.</span><span style="color:#a6e22e">setName</span><span style="color:#f92672">(</span><span style="color:#e6db74">&#34;张三&#34;</span><span style="color:#f92672">);</span>
        user<span style="color:#f92672">.</span><span style="color:#a6e22e">setId</span><span style="color:#f92672">(</span><span style="color:#e6db74">&#34;HO123&#34;</span><span style="color:#f92672">);</span>
        
        List<span style="color:#f92672">&lt;</span>Role<span style="color:#f92672">&gt;</span> list <span style="color:#f92672">=</span> <span style="color:#66d9ef">new</span> ArrayList<span style="color:#f92672">&lt;</span>Role<span style="color:#f92672">&gt;();</span>
        <span style="color:#66d9ef">for</span><span style="color:#f92672">(</span><span style="color:#66d9ef">int</span> i<span style="color:#f92672">=</span>1<span style="color:#f92672">;</span>i<span style="color:#f92672">&lt;=</span>2<span style="color:#f92672">;</span>i<span style="color:#f92672">++)</span> <span style="color:#f92672">{</span>
        	Role role <span style="color:#f92672">=</span> <span style="color:#66d9ef">new</span> Role<span style="color:#f92672">();</span>
            role<span style="color:#f92672">.</span><span style="color:#a6e22e">setId</span><span style="color:#f92672">(</span><span style="color:#e6db74">&#34;R&#34;</span> <span style="color:#f92672">+</span> Integer<span style="color:#f92672">.</span><span style="color:#a6e22e">toString</span><span style="color:#f92672">(</span>i<span style="color:#f92672">));</span>
            role<span style="color:#f92672">.</span><span style="color:#a6e22e">setName</span><span style="color:#f92672">(</span><span style="color:#e6db74">&#34;经办&#34;</span><span style="color:#f92672">);</span>
            list<span style="color:#f92672">.</span><span style="color:#a6e22e">add</span><span style="color:#f92672">(</span>role<span style="color:#f92672">);</span>
        <span style="color:#f92672">}</span>
        
        user<span style="color:#f92672">.</span><span style="color:#a6e22e">setRoleList</span><span style="color:#f92672">(</span>list<span style="color:#f92672">);</span>
        
        <span style="color:#75715e">//protobuf序列化
</span><span style="color:#75715e"></span>        <span style="color:#66d9ef">long</span> proto1 <span style="color:#f92672">=</span> <span style="color:#f92672">(</span><span style="color:#66d9ef">new</span> Date<span style="color:#f92672">()).</span><span style="color:#a6e22e">getTime</span><span style="color:#f92672">();</span>
        LinkedBuffer buffer <span style="color:#f92672">=</span> LinkedBuffer<span style="color:#f92672">.</span><span style="color:#a6e22e">allocate</span><span style="color:#f92672">(</span>LinkedBuffer<span style="color:#f92672">.</span><span style="color:#a6e22e">DEFAULT_BUFFER_SIZE</span><span style="color:#f92672">);</span>
        Schema<span style="color:#f92672">&lt;</span>User<span style="color:#f92672">&gt;</span> schema <span style="color:#f92672">=</span> RuntimeSchema<span style="color:#f92672">.</span><span style="color:#a6e22e">getSchema</span><span style="color:#f92672">(</span>User<span style="color:#f92672">.</span><span style="color:#a6e22e">class</span><span style="color:#f92672">);</span>
        ProtobufIOUtil<span style="color:#f92672">.</span><span style="color:#a6e22e">toByteArray</span><span style="color:#f92672">(</span>user<span style="color:#f92672">,</span> schema<span style="color:#f92672">,</span> buffer<span style="color:#f92672">);</span>
        <span style="color:#66d9ef">byte</span><span style="color:#f92672">[]</span> serializerResult <span style="color:#f92672">=</span> ProtoBufUtil<span style="color:#f92672">.</span><span style="color:#a6e22e">serializer</span><span style="color:#f92672">(</span>user<span style="color:#f92672">);</span>
 
        System<span style="color:#f92672">.</span><span style="color:#a6e22e">out</span><span style="color:#f92672">.</span><span style="color:#a6e22e">println</span><span style="color:#f92672">(</span><span style="color:#e6db74">&#34;protobuf序列化二进制:&#34;</span> <span style="color:#f92672">+</span> bytes2hex<span style="color:#f92672">(</span>serializerResult<span style="color:#f92672">));</span>
        System<span style="color:#f92672">.</span><span style="color:#a6e22e">out</span><span style="color:#f92672">.</span><span style="color:#a6e22e">println</span><span style="color:#f92672">(</span><span style="color:#e6db74">&#34;protobuf序列化ascii:&#34;</span> <span style="color:#f92672">+</span> <span style="color:#66d9ef">new</span> String<span style="color:#f92672">(</span>serializerResult<span style="color:#f92672">));</span>
        System<span style="color:#f92672">.</span><span style="color:#a6e22e">out</span><span style="color:#f92672">.</span><span style="color:#a6e22e">println</span><span style="color:#f92672">(</span><span style="color:#e6db74">&#34;protobuf序列化字节长度:&#34;</span> <span style="color:#f92672">+</span> serializerResult<span style="color:#f92672">.</span><span style="color:#a6e22e">length</span><span style="color:#f92672">);</span>
        User protouser <span style="color:#f92672">=</span> <span style="color:#66d9ef">new</span> User<span style="color:#f92672">();</span>
        ProtobufIOUtil<span style="color:#f92672">.</span><span style="color:#a6e22e">mergeFrom</span><span style="color:#f92672">(</span>serializerResult<span style="color:#f92672">,</span> protouser<span style="color:#f92672">,</span> schema<span style="color:#f92672">);</span>
 
        <span style="color:#66d9ef">long</span> proto2 <span style="color:#f92672">=</span> <span style="color:#f92672">(</span><span style="color:#66d9ef">new</span> Date<span style="color:#f92672">()).</span><span style="color:#a6e22e">getTime</span><span style="color:#f92672">();</span>
        System<span style="color:#f92672">.</span><span style="color:#a6e22e">out</span><span style="color:#f92672">.</span><span style="color:#a6e22e">println</span><span style="color:#f92672">(</span><span style="color:#e6db74">&#34;protobuf反序列化结果:&#34;</span> <span style="color:#f92672">+</span> protouser<span style="color:#f92672">);</span>
        System<span style="color:#f92672">.</span><span style="color:#a6e22e">out</span><span style="color:#f92672">.</span><span style="color:#a6e22e">println</span><span style="color:#f92672">(</span><span style="color:#e6db74">&#34;protobuf序列化耗时:&#34;</span> <span style="color:#f92672">+</span> <span style="color:#f92672">(</span>proto2 <span style="color:#f92672">-</span> proto1<span style="color:#f92672">));</span>
        
    <span style="color:#f92672">}</span>
 
<span style="color:#f92672">}</span>
</code></pre></div><p>运行后结果如下</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">protobuf序列化二进制:0A <span style="color:#ae81ff">05</span> <span style="color:#ae81ff">48</span> 4F <span style="color:#ae81ff">31</span> <span style="color:#ae81ff">32</span> <span style="color:#ae81ff">33</span> <span style="color:#ae81ff">12</span> <span style="color:#ae81ff">06</span> E5 BC A0 E4 B8 <span style="color:#ae81ff">89</span> <span style="color:#ae81ff">18</span> AC <span style="color:#ae81ff">02</span> <span style="color:#ae81ff">22</span> <span style="color:#ae81ff">06</span> E5 A4 <span style="color:#ae81ff">87</span> E6 B3 A8 2A 0C 0A <span style="color:#ae81ff">02</span> <span style="color:#ae81ff">52</span> <span style="color:#ae81ff">31</span> <span style="color:#ae81ff">12</span> <span style="color:#ae81ff">06</span> E7 BB 8F E5 8A 9E 2A 0C 0A <span style="color:#ae81ff">02</span> <span style="color:#ae81ff">52</span> <span style="color:#ae81ff">32</span> <span style="color:#ae81ff">12</span> <span style="color:#ae81ff">06</span> E7 BB 8F E5 8A 9E 
protobuf序列化ascii:
HO123张三�<span style="color:#e6db74">&#34;备注*
</span><span style="color:#e6db74">R1经办*
</span><span style="color:#e6db74">R2经办
</span><span style="color:#e6db74">protobuf序列化字节长度:54
</span><span style="color:#e6db74">protobuf反序列化结果:name=张三,id=HO123,age=300,role1=R1,role2=R2
</span><span style="color:#e6db74">protobuf序列化耗时:185
</span></code></pre></div><p>protobuf报文在实际传输中是以二进制方式传输的，这里为了方便分析，我把它专成了ascii字符。可以看到，protobuf的压缩效率非常高，除了基本的字段内容之外，其他的标签之类的全部都压缩了，以二进制方式存储，所以它的报文格式会比xml小非常多，当然代价就是可读性没有那么友好，这也是为什么谷歌要定义proto文件的原因。在没有这个文件的情况下，你几乎不可能看懂这个报文所表示的数据结构。</p>
<h3 id="性能测试">性能测试</h3>
<p>说了那么多，现在来看看实际protobuf能为我们带来多少传输报文的性能提升。这里我选取了目前日常使用最普遍的XML以及json格式，这两个是文本形式的序列化框架，以及hessian2，这个是老牌的二进制序列化框架，也是dubbo里默认的序列化方案。几种格式都可以做到跨语言，现在我们来看看他们的性能差距到底有多少。</p>
<h4 id="测试基准环境">测试基准环境：</h4>
<ul>
<li>硬件资源：macbook pro 笔记本(2012 late,core i5 2.5GHz,8G内存）</li>
<li>java版本：jdk1.8</li>
<li>protobuf序列化框架：protostuff</li>
<li>xml序列化框架：jdk原生jaxb</li>
<li>json序列化框架：fastjson</li>
<li>hessian序列化框架：hessian</li>
</ul>
<h4 id="测试方法">测试方法</h4>
<p>将<code>List&lt;Role&gt;</code>循环多次，模拟不同的报文size大小，以此测试序列化框架的性能。</p>
<p>Round 1:<code>size=10</code></p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">protobuf序列化字节长度:167
protobuf反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
protobuf序列化耗时:196

xml序列化字节长度:645
xml反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
xml序列化耗时:149

json序列化字节长度:350
json反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
json序列化耗时:223

hessian2序列化字节长度:690
hessian2反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
hessian2序列化耗时:63
</code></pre></div><p>可以看出在数据量很小的情况，几个框架的性能差别不大，hessian2最好，xml甚至比protobuf还要高一些。但是有一个地方需要注意，就是序列化后的字节长度，json是xml的大概1/2多一点，而protobuf只有xml的1/4，而hessian甚至比xml还要长。这在存储敏感或者带宽敏感的场景下是至关重要的。之后的所有测试存储所占空间的比例基本都是一样。</p>
<p>Roune2:<code>size=1000</code></p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">protobuf序列化字节长度:15919
protobuf反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
protobuf序列化耗时:211

xml序列化字节长度:53027
xml反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
xml序列化耗时:256

json序列化字节长度:29962
json反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
json序列化耗时:249

hessian2序列化字节长度:60992
hessian2反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
hessian2序列化耗时:114
</code></pre></div><p>当数据量来到1000这个量级，仍然是hessian一马当先，可以看到protobuf已经反超xml了，这时候json的性能也已经超过xml，xml的劣势开始渐渐显现。</p>
<p>Round3:<code>size=100000</code></p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">protobuf序列化字节长度:1788921
protobuf反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
protobuf序列化耗时:333

xml序列化字节长度:5489029
xml反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
xml序列化耗时:1059

json序列化字节长度:3188964
json反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
json序列化耗时:726

hessian2序列化字节长度:6288994
hessian2反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
hessian2序列化耗时:785

</code></pre></div><p>来到10万这个量级，protobuf反超！而且优势非常明显，几乎是xml性能的3倍，hessian的2倍，序列化后的字节大小为1.7M左右，xml的1/4</p>
<p>Round4:<code>size=1000000</code></p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">protobuf序列化字节长度:18888922
protobuf反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
protobuf序列化耗时:2405

xml序列化字节长度:55889030
xml反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
xml序列化耗时:4087

json序列化字节长度:32888965
json反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
json序列化耗时:3063

hessian2序列化字节长度:63888995
hessian2反序列化结果:name<span style="color:#f92672">=</span>张三,id<span style="color:#f92672">=</span>HO123,age<span style="color:#f92672">=</span>300,role1<span style="color:#f92672">=</span>R1,role2<span style="color:#f92672">=</span>R2
hessian2序列化耗时:4747
</code></pre></div><p>数据量为100万条，此时protobuf的报文数据大小为18M，xml已经达到了55M，json也有33M，hessian更是到了64M，protobuf的速度仍然是xml的差不多两倍。这里比较诡异的是json在好几次测试中耗时只有1000多ms，但多测几次又会变回3000多ms，从生成的字节码来看，3000ms是比较合理的结果，不知道是做了什么样子的优化。</p>
<h3 id="结论">结论</h3>
<p>在报文数据量很小的情况，几种格式差别不是很大，建议都可以选择。如果对于报文解析性能要求很高，报文体积小的情况下可以选择hessian，体积大选protobuf。如果需要文本格式，选择json。如果对于存储或者带宽敏感，建议选择protobuf，体积比其他几种格式小太多，传输或者存储都很方便。</p>
<h3 id="补充说明">补充说明</h3>
<p>看了protobuf的官方文档，感觉protobuf的性能还有提高的空间，就用原生的protobuf测试了一下，静态编译了<code>user.proto</code>文件，测试结果如下</p>
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">数据量大小:10
原生protobuf序列化字节长度:167
原生protobuf序列化耗时:54

数据量大小:1000
原生protobuf序列化字节长度:15919
原生protobuf序列化耗时:86

数据量大小:100000
原生protobuf序列化字节长度:1788921
原生protobuf序列化耗时:289

数据量大小:1000000
原生protobuf序列化字节长度:18888922
原生protobuf序列化耗时:2371
</code></pre></div><p>可以看出性能有了很大的提高，在各阶段的测试中都是最快的。对比编码后的字节数，和protostuff是完全一致的，说明两种框架最终出来的结果是一样的，但是原生的protobuf要快了很多，看来果然鱼和熊掌不可兼得，运行时的编译对于性能相比静态编译还是有相当的损耗的。在小于1000的数据量级，两者的性能差距差不多有3倍。对于极致追求性能的场景，还是建议使用原生的protobuf。当然本次测试结果还没有达到官方宣称的比xml要快20～100倍这个数量级，这个估计需要C++或者go的环境下才能实现，留给其他同学进行测试了。</p>
<p><a href="https://github.com/shinji3887/protobuftest">测试用到的github代码地址</a></p>

        
                
        
              <hr>
              <ul class="pager">
                  
                  <li class="previous">
                      <a href="/post/restful-api-design/" data-toggle="tooltip" data-placement="top" title="浅淡RESTful api设计规范">&larr; Previous Post</a>
                  </li>
                  
                  
                  <li class="next">
                      <a href="/post/http2-introduce/" data-toggle="tooltip" data-placement="top" title="HTTP/2协议介绍">Next Post &rarr;</a>
                  </li>
                  
              </ul>
  
              

<div id="gitalk-container"></div>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/gitalk@1/dist/gitalk.css">
<script src="https://cdn.jsdelivr.net/npm/gitalk@1/dist/gitalk.min.js"></script>
<script src="/js/md5.min.js"></script>
<script>
	const gitalk = new Gitalk({
	  clientID: 'aff2580d8cc58af83367',
	  clientSecret: '747547f5f87fcc5145b847ab76a498d7e501319f',
	  repo: 'comment',
	  owner: 'shinji3887',
	  admin: ['shinji3887'],
	  id: md5(location.pathname),      
	  distractionFreeMode: false  
	})

	gitalk.render('gitalk-container')
</script>


            </div>
            
            <div class="
                col-lg-8 col-lg-offset-2
                col-md-10 col-md-offset-1
                sidebar-container">

                
                <section>
                    <hr class="hidden-sm hidden-xs">
                    <h5><a href="/tags/">FEATURED TAGS</a></h5>
                    <div class="tags">
                     
                    
                        
                            <a href="/tags/api-gateway" title="api-gateway">
                                api-gateway
                            </a>
                        
                    
                        
                    
                        
                            <a href="/tags/cloud-native" title="cloud-native">
                                cloud-native
                            </a>
                        
                    
                        
                            <a href="/tags/devops" title="devops">
                                devops
                            </a>
                        
                    
                        
                            <a href="/tags/docker" title="docker">
                                docker
                            </a>
                        
                    
                        
                    
                        
                    
                        
                    
                        
                    
                        
                            <a href="/tags/istio" title="istio">
                                istio
                            </a>
                        
                    
                        
                    
                        
                            <a href="/tags/kubernetes" title="kubernetes">
                                kubernetes
                            </a>
                        
                    
                        
                            <a href="/tags/microservice" title="microservice">
                                microservice
                            </a>
                        
                    
                        
                    
                        
                    
                        
                    
                        
                    
                        
                            <a href="/tags/restful" title="restful">
                                restful
                            </a>
                        
                    
                        
                    
                        
                            <a href="/tags/servicemesh" title="servicemesh">
                                servicemesh
                            </a>
                        
                    
                        
                            <a href="/tags/spring-cloud" title="spring-cloud">
                                spring-cloud
                            </a>
                        
                    
                        
                            <a href="/tags/vue" title="vue">
                                vue
                            </a>
                        
                    
                        
                    
                        
                    
                    </div>
                </section>

                
                <hr>
                <h5>FRIENDS</h5>
                <ul class="list-inline">
                    
                        <li><a target="_blank" href="https://skyao.io/">小剑的博客</a></li>
                    
                        <li><a target="_blank" href="https://zhaohuabing.com/">huabing的博客</a></li>
                    
                        <li><a target="_blank" href="http://blog.didispace.com/">程序猿DD的博客</a></li>
                    
                </ul>
            </div>
        </div>
    </div>
</article>




<footer>
    <div class="container">
        <div class="row">
            <div class="col-lg-8 col-lg-offset-2 col-md-10 col-md-offset-1">
                <ul class="list-inline text-center">
                   
                   <li>
                       <a href="" rel="alternate" type="application/rss+xml" title="L&#39;s Blog" >
                           <span class="fa-stack fa-lg">
                               <i class="fa fa-circle fa-stack-2x"></i>
                               <i class="fa fa-rss fa-stack-1x fa-inverse"></i>
                           </span>
                       </a>
                   </li>
                   
                    
                    <li>
                        <a href="mailto:18016380795@163.com">
                            <span class="fa-stack fa-lg">
                                <i class="fa fa-circle fa-stack-2x"></i>
                                <i class="fa fa-envelope fa-stack-1x fa-inverse"></i>
                            </span>
                        </a>
                    </li>
		    
                    
                    
                    
                    

                    

		    
                    
                    <li>
                        <a target="_blank" href="/link%20of%20wechat%20QR%20code%20image">
                            <span class="fa-stack fa-lg">
                                <i class="fa fa-circle fa-stack-2x"></i>
                                <i class="fa fa-wechat fa-stack-1x fa-inverse"></i>
                            </span>
                        </a>
                    </li>
		    
                    
                    <li>
                        <a target="_blank" href="https://github.com/shinji3887">
                            <span class="fa-stack fa-lg">
                                <i class="fa fa-circle fa-stack-2x"></i>
                                <i class="fa fa-github fa-stack-1x fa-inverse"></i>
                            </span>
                        </a>
                    </li>
		    
                    
                    <li>
                        <a target="_blank" href="https://www.linkedin.com/in/lupeier">
                            <span class="fa-stack fa-lg">
                                <i class="fa fa-circle fa-stack-2x"></i>
                                <i class="fa fa-linkedin fa-stack-1x fa-inverse"></i>
                            </span>
                        </a>
                    </li>
		    
                </ul>
		<p class="copyright text-muted">
                    Copyright &copy; L&#39;s Blog , 2018
                    <br>
                    <br>
                    <a href="http://icp.chinaz.com/info?q=lupeier.com" target="_blank">备案号：沪ICP备19022667号-1</a>                    
                </p>
            </div>
        </div>
    </div>
</footer>



<script>
    function async(u, c) {
      var d = document, t = 'script',
          o = d.createElement(t),
          s = d.getElementsByTagName(t)[0];
      o.src = u;
      if (c) { o.addEventListener('load', function (e) { c(null, e); }, false); }
      s.parentNode.insertBefore(o, s);
    }
</script>






<script>
    
    if($('#tag_cloud').length !== 0){
        async("/js/jquery.tagcloud.js",function(){
            $.fn.tagcloud.defaults = {
                
                color: {start: '#bbbbee', end: '#0085a1'},
            };
            $('#tag_cloud a').tagcloud();
        })
    }
</script>


<script>
    async("/js/fastclick.js", function(){
        var $nav = document.querySelector("nav");
        if($nav) FastClick.attach($nav);
    })
</script>


<script>
    (function(){
        var bp = document.createElement('script');
        var curProtocol = window.location.protocol.split(':')[0];
        if (curProtocol === 'https'){
       bp.src = 'https://zz.bdstatic.com/linksubmit/push.js';
      }
      else{
      bp.src = 'http://push.zhanzhang.baidu.com/push.js';
      }
        var s = document.getElementsByTagName("script")[0];
        s.parentNode.insertBefore(bp, s);
    })();
</script>







</body>
</html>
